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Abstract 


Early evaluation of the architecture of a system or a product line of systems is a low-cost risk 
reduction method for determining whether the system(s) will achieve its business and quality 
goals. The Architecture Tradeoff Analysis Method (ATAM) is an architecture evaluation tech- 
nique currently evolving at the Software Engineering Institute (SEI). The input to the ATAM 
consists of a system or product line architecture and the perspectives of stakeholders involved 
with that system or product line. The output of the ATAM is (1) a collection of scenarios that 
help specify the context of the system’s or product line’s use and the product line’s evolution, 
(2) improved architectural documentation (usually), and (3) analysis results (in particular, a set 
of issues to consider, risks, and potential sensitivity and tradeoff points within the architecture). 
Currently, there are no generally accepted industry-wide standards for describing a system ar- 
chitecture, and ATAM evaluations are often tailored to the available documentation. 


The Architectures Directorate of the C4I Integration Support Activity (CISA), Office of the As- 
sistant Secretary of Defense for Command, Control, Communications, and Intellegence 
(OASD[C3I]) has defined a framework for architecture development, presentation, and integra- 
tion to be used across the military services and defense agencies. This framework for Com- 
mand, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance 
(C4ISR) is becoming the required method for describing information systems within the De- 
partment of Defense (DoD) and other U.S. Government agencies. This report describes how 
various C4ISR products can be used in the context of an ATAM evaluation and their relative 
value for generating quality attribute-specific scenarios required for an ATAM evaluation. 
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1 The Architecture Tradeoff Analysis 
Method (ATAM) 


The Architecture Tradeoff Analysis Method (ATAM) is an architecture evaluation technique 
currently evolving at the Software Engineering Institute (SEI). The input to the ATAM con- 
sists of a system or product line architecture and the perspectives of stakeholders involved 
with that system or product line. The ATAM relies on generating use-case and change-case 
scenarios to assess the architecture. Use cases and change cases are described below: 


e Use cases represent operational conditions that the architecture should support and dem- 
onstrate the effectiveness of the architecture to satisfy these operating conditions. 


¢ Change cases represent expected system modifications that may cause architectural 
changes and demonstrate how efficiently the architecture can be modified. 


The ATAM achieves its architecture evaluation for quality-attribute goals by using an under- 
standing of the architectural mechanisms that are used to achieve particular quality goals and 
the implications of those mechanisms. Each quality attribute is examined from the point of 
view of three perspectives: what external stimuli causes the architecture to respond or change, 
what mechanisms are used within the architecture to control the response, and how the 
response to these stimuli is measured. For performance, for example, the external stimuli are 
events arriving at the system; the mechanisms are scheduling, concurrency, and resource man- 
agement; and the measurements are latency or throughput. For modifiability, for example, the 
external stimuli are changes to the system, the mechanisms for controlling the cost of changes 
are encapsulation and data indirection, and the measurement is the cost of a collection of 
changes. 


The ATAM consists of a number of steps, starting with a presentation of the business drivers, 
the architecture, and the identification of architecture styles in the system. This is followed by 
the generation of scenarios, prioritization of the scenarios, and mapping the prioritized scenar- 
ios onto the architecture styles. | 


The ATAM uses stakeholder perspectives to derive a collection of scenarios giving specific 
instances for usage, performance requirements or growth, various types of failures, various 
possible threats, and various likely modifications. The scenarios are used for the evaluators to 
understand the action of the system or product line. The evaluation results in three different 


types of output: 
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a collection of “sensitivity” or “tradeoff” points. A sensitivity point is a component or 
decision made in the architecture that is critical for the achievement of a particular quality 
attribute. A tradeoff point is a sensitivity point that is sensitive for multiple quality 
attributes 


a framework for reasoning about the system or product line. The framework for reasoning 
about the system or product line may take a variety of forms. It may be the discussion that 
follows from the exploration of a scenario, it may be a model or a portion of a model and a 
discussion of how that model might be analyzed when instantiated, or it may be a formula 
that represents how to calculate a value of a particular quality attribute. 


a list of issues, or decisions not yet made. The list of issues or decisions not yet made 
arises from the stage of the life cycle of the evaluation. An architecture represents a collec- 
tion of decisions. Not all decisions may have been made, even when designing the archi- 
tecture. Some of these decisions are known to the development team as having not been 
made and are on a list for further consideration. Others are unknown to the development 
team and stakeholders. 


The only ATAM step affected by the C4ISR! Framework products is the scenario generation. 
The other steps in the evaluation phase are not affected by the C4ISR, since they assume only 
that some architectural representations will be available on which to map the scenarios. 


1. 


C4ISR is Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance. 
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2 C4ISR Architecture Framework 


The C4ISR Architecture Framework defines three views (operational, system, and technical) 
and two classes of products (essential and supporting). The three views are defined in the Sec- 
tions 2.1.1 through 2.1.3 of the framework document and are summarized below: 


1. The operational architecture view is a description of the tasks and activities, operational 
elements, and information flows required to accomplish or support a military operation. It 
contains descriptions (often graphical) of the operational elements, assigned tasks and 
activities, and information flows required to support the war-fighter. It defines the types of 
information exchanged, the frequency of exchange, which tasks and activities are sup- 
ported by the information exchanges, and the nature of information exchanges in detail 
sufficient to ascertain specific interoperability requirements. 


2. The systems architecture view is a description, including graphics, of systems and inter- 
connections providing for, or supporting, war-fighting functions. For a domain, the sys- 
tems architecture view shows how multiple systems link and interoperate, and it may 
describe the internal construction and operations of particular systems within the architec- 
ture. For the individual system, the systems architecture view includes the physical con- 
nection, location, and identification of key nodes (including materiel item nodes), circuits, 
networks, war-fighting platforms, etc., and it specifies system and component perfor- 
mance parameters (e.g., mean time between failure, maintainability, availability). The sys- 
tems architecture view associates physical resources and their performance attributes to 
the operational view and its requirements per standards defined in the technical architec- 
ture. 


3. The technical architecture view is the minimal set of rules governing the arrangement, 
interaction, and interdependence of system parts or elements, whose purpose is to ensure 
that a conforming system satisfies a specified set of requirements. The technical architec- 
ture view provides the technical systems-implementation guidelines upon which engineer- 
ing specifications are based, common building blocks are established, and product lines 
are developed. The technical architecture view includes a collection of the technical stan- 
dards, conventions, rules, and criteria organized into profile(s) that govern system ser- 
vices, interfaces, and relationships for particular systems architecture views and that relate 
to particular operational views. 


The framework specifies that the essential products must be supplied, and different systems 
could require different supporting products depending on the context. However, it does not 
provide more specific guidance regarding what supporting products might be needed: 
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The decision of which products to build, beyond the essential set, must be made 
based on the issues the architecture is intended to explore and the resulting 
characteristics that the architecture must capture. A given architecture may 
contain all of the supporting products, a selected subset, or none of the support- 
ing products. For example, an architecture that is to be used in business process 
reengineering should include an Activity Model; an architecture that is to be 
used in examining technology insertion and achievable states of “to-be” capa- 
bilities should include a System Technology Forecast [C4ISR Architecture 
Framework Version 2, Section 3.1.2.2]. 


The situation is further complicated because the framework does not provide a process for 
generating the products. Thus, an organization developing an architecture that is compliant 
with the C4ISR Framework could be faced with an unbounded amount of effort. 


The Systems Architectures Laboratory at George Mason University has developed a C4ISR 
Architecture Framework Implementation that includes a process for generating the essential 
and supporting products for the operational and systems architecture views. The process takes 
a systems engineering point of view and identifies aes between the products that lead 
to a six-step process: 


1. STEP 0: Problem definition and collection of domain information 
STEP 1; Operational concept and requirements 


STEP 2: Functions and organizations 


= oe ἸῺ 


STEP 3: Activity model, logical data model, need lines, system nodes, system elements 
and functions, and task allocation 


5. STEP 4: Operational information elements and exchanges, system functionality descrip- 
tion, physical data model 


6. STEP 5: System information elements and exchanges, local area networks/wide area net- 
works (LAN/WANs), system interface descriptions, system performance 


This process is shown in Figure 2-1, which was reproduced from the AFCEA! Course 503E, 
Chapter 3, Page 4. 


1. | AFCEA is the Armed Forces Communications and Electronics Association. 
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The process is complex because there is tight coupling among products, and the number of 
relationships increase dramatically with the layers of representation in the activity model (OV- 
2) and the system functionality description (SV-4). However, the process produces products in 
all six steps, allowing for a continuous review and evaluation of the architecture description 
process. ! | 

This report does not deal with product generation; we assume that some collection of products 
has been developed. Instead, we focus on the task of evaluating the architecture described by 
the products. The preceding discourse simply points out that the effort required to generate the 
products is not trivial; therefore, it is important to choose wisely. As we shall see, depending 
on the combination of products available at the time of the evaluation, different types of 
attribute-specific scenarios can be generated. Thus, the choice of products could be driven by 
the quality attributes of interest. 


One reason for the complexity of relationships between the operational and system views is 
that the operational activities can be developed quite separately from the system’s functions. 
Hence a general activity can involve a number of system functions, and a system function can 
perform parts of many activities, leading to the complex relationships. The complexity can be 
reduced by carefully choosing the activities to correspond to the system functions. 


The techniques used in the C4ISR Framework derive from the structured analysis tradition. 
Over the past 10 years, object-oriented methodologies have become popular, but object orien- 
tation forces the developers to consider only relationships between objects and removes the 
“activity” versus “function” split. The Unified Modeling Language (UML) supports the devel- 
opment of object-oriented views and is becoming an industry standard. UML products, how- 
ever, are not interchangeable with C4ISR products since they are developed with a different 
mindset (structure analysis versus object orientation). The use of similar names for different 
concepts in the two traditions makes it difficult to know what is meant by a particular term. 


1. Alexander Levis and his team at the Systems Architectures Laboratory, George Mason University have built an executable 
model of the C4ISR products. Such an executable mode! could be used to validate the C4ISR products, by developing, ex- 
ecuting, and analyzing many test cases. 
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3 Linking the C4ISR Architecture Frame- 
work and the ATAM 


The C4ISR Framework concentrates on view and product definitions, and it does not require 
use cases or change cases to clarify and assess the effectiveness of the architecture. The ATAM 
Evaluation phase relies on generating use-case and change-case scenarios to analyze the archi- 
tectures. It is not immediately obvious which types of use cases and change cases can or can 
not be analyzed against which architectural views and products. This is further complicated by 
the fact that the framework allows the products to be enhanced by annotations or by innovative 
usage. Products may even be replaced by alternative products that are deemed to be equivalent 
to the C4ISR Framework products. 


3.1 C4ISR Products as Potential Sources of 
Scenarios 


We reviewed the descriptions of the C4ISR products in Section 4 of the framework and the 
product attribute tables defined in Appendix A of the C4ISR Architecture Framework Version 
2, and we determined how could they be used to generate scenarios for use cases and change 
cases associated with performance, dependability, security, modifiability, and interoperability. 
In doing so, we assumed a strict interpretation of the appendices. ! Our detailed observations 
are listed in Appendix A and are summarized in Table 3-1 and Table 3-2. The tables indicates 
what attribute-specific information is contained in the product, information which could be 
used to generate attribute-specific scenarios. 


1. Obviously more could be done with any view that is made richer by annotations or innovation. 
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Possible sources of 
attribute-specific 
scenarios from 
individual products 


Modifiability/ 
Interoperabili 


Overview and Summary Informa- 
tion (AV-1) 


|| Dependability 


Integrated Dictionary (AV-2) 


High-Level Operational Concept throughput points of exposure 

Graphic (OV-1) latency failure ' | intersection 
availability of spoofing 
assets 


Operational Node Connectivity 
Description (OV-2) 


Operational Information Exchange message size redundancy Classification of | protocols 
Matrix (OV-3) media messages formats 
frequency of LISI levels* 
exchanges 
timeliness 
throughput 


System Interface Description (SV-1) | throughput redundancy of standards 
channel components/ protocols 

Capacity functions components 

latency contained 
functions 

performed 


Technical Architecture Profile (TV- 
1) 


Table 3-1: Summary of Essential Products 


a. LIS! is fevels of information systems interoperability. 
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Possible sources of 
attribute-specific 


scenarios from 
individual products 


Command Relationships Chart (OV- 
4) 
Activity Model (OV-5) 


Operational Activity Sequence and alternative paths points of 
Timing Descriptions (OV-6a, 6b, throughput failure 
and 6c) latency 


Systems Communications Descrip- WAN/LAN fault rates 
tion (SV-2) characteristics 


steps in path 
Systems” Matrix (SV-3) 
Systems Functionality Description 
(SV-4) 


Operational Activity to System 
Function Traceability Matrix (SV-5) 


4 
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activities 
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classification 


encryption 


functionality evo- 
ution 


classification 


media 
format 


System Information Exchange classification 


Matrix (SV-6) 


frequency 


timeliness 
throughput 


data rates 
throughput 
capacity 
initialization/ 
restart time 
response time 


8) 


System Technology Forecast (SV-9) a 
System Activity Sequence and Tim- | execution paths points of 
failure 


System Performance Parameters 
Matrix (SV-7) 


upgraded 
functionality 


upgraded 
capabilities 


Ὄ 
i 


removing 


ing Descriptions (SV-10a, 10b, and event times attack constraints on 


10c) 


Physical Data Model (SV-11) _ message size file system 
properties 

Standards Technology Forecast 

(TV-2) 


Table 3-2: Summary of Supporting Products 


functionality 


new standards 
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3.2 C4ISR Products and ATAM Screening Questions 


During scenario gathering, the evaluators use screening questions to elicit use-case and 
change-case scenarios from the stakeholders, and they prioritize these cases to determine 
which ones will be used for analysis. The high-priority scenarios will be a mixture of scenar- 
ios, some that are strictly operational and some that are system oriented. 


The screening questions in an ATAM are targeted at what is equivalent to C4ISR system-level 
views rather than C4ISR operational views. This section describes how these questions could 
be extended to include questions targeted at the C4ISR operational views. Appendix B lists 
example screening questions that are appropriate for operational views, and Appendix C lists 
examples of actual ATAM scenarios. 


There are two types of C4ISR operational view screening questions: those concerning the 
operational environment, which help us to derive an operational context for the developing 
detailed use-case scenarios, and those concerned with the logistics (support) environment, 
which help us to derive a logistics context for detailed change-case scenarios. 


3.2.1 Operational Environment 


The intent of the screening questions (contained in Appendix B.1) is to move from a very 
wide-scale operational problem set to a much smaller scale problem set that is likely to expose 
the architectural weaknesses and strengths. They should result in focusing the use cases. 


3.2.1.1 Mission and Geography 


These questions (contained in Appendix B.1.1) help us to focus on those issues that are stress- 
ful to the operational architecture and those that are difficult to accomplish. Information 
uncovered by these questions can also create a general background environmental “workload” 
that represents those conditions. A workload could, for example, indicate the number of ves- 
sels and aircraft within the area of interest (AOI), describe how an operational picture is 
achieved and distributed to the platforms and command centers involved with the mission, and 
show the acceptable latency for upgrades of this picture. 


3.2.1.2 Organization and Command Structure 


These questions (contained in Appendix B.1.2) help us to establish the command structures 
involved and how the structures interact to conduct the missions. These questions could lead 
us to understand which missions could have command-structure interactions that are problem- 
atic, or which missions will have command-structure problems when failures arise. 


eee 
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3.2.1.3 Operational Abstract Quality Attributes 


The operational environments established by the previous screening questions can now be 
used to develop abstract scenarios for the appropriate quality attributes, such as availability, 
performance, security, and interoperability. For each such quality attribute, there are seeding 
questions (contained in Appendix B.1.3) that will lead to some abstract scenarios. To generate 
the scenarios, the architect will have to reference some high-level operational views of the sys- 
tem. 


3.2.2 Logistics Environment 


The screening questions (contained in Appendix B.2) will help us focus the problems with sys- 
tem upgrades and/or logistics. They should lead to an understanding of the strategies involved 
in these issues and allow us to focus the change-case scenarios appropriately. 


3.2.2.1. Platform and Command Center Upgrades 


These questions (contained in Appendix B.2.1) help us to establish the ground rules under 
which platform and/or command-center upgrades will be made. For example, how are classes 
of cutters and aircraft upgraded? Are cutters upgraded during routine maintenance operations, 
or can certain upgrades be done less intrusively? Are all aircraft of a certain class removed 
from service, their crews retrained, and then all restored to service? The screening questions 
should lead to a basic understanding of these ground rules for platform and command-center 
upgrades. 


A related issue is that of maintaining platform configurations. How much variety is allowed 
between platforms in the same and different operational areas? 


3.2.3 Evolution Strategy 


Many of the infrastructure components within certain systems become obsolete in relatively 
short time periods. There can be a strategy of extending the life of these components beyond 
its normal range or a strategy of timely replacement of components. With software, the prob- 
lem is compounded due to the relatively short lifetime of software components. These ques- 
tions (contained in Appendix B.2.2) will help us to focus the scenarios. 


3.2.4 Legacy Systems 


Many of the legacy systems will remain until retirement, and in some cases they must interop- 
erate with the new systems being introduced. The screening questions (contained in Appendix 
B.2.3) in this area should lead to an understanding of the “ground rules” under which these 
operations will occur. 
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3.2.5 Logistics Abstract Quality Attributes 


The results of the above screening process should allow us to capture the major issues to be 
considered for the change cases. The questions (contained in Appendix B.2.4) will help us to 
generate change cases for the quality attributes of modifiability and interoperability. The pro- 
cess here parallels that of the use cases, concerning levels of abstraction and detail in both the 
change-case scenarios and the view that are referenced. 


a ee ee -, ee eb 
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4 Conclusions and Recommendations 


The product summaries in Table 3-1 and Table 3-2 suggest how each product could contribute 
to the development of quality attribute-specific scenarios by using the ATAM screening ques- 


tions. 


The products are complementary with respect to quality attributes, and different combinations 
of products could be used to generate different scenarios, depending on the attributes of inter- 
est. For example, Table 4-1 show the consolidation of all the essential products, the supporting 
operational view products, and selected products focused on system evolution. Any number of 
these combinations could be used as guidance to identify products that are desirable for gener- 


ating attribute-specific scenarios. 


Possible sources of attribute- 


specific scenarios from product 


combinations 


Essential Products 
Overview and Summary Information (AV-1) 
Integrated Dictionary (AV-2) 


High-Level Operational Concept Graphic (OV-1) 
Operational Node Connectivity Description (OV-2) 
Operational Information Exchange Matrix (OV-3) 


System Interface Description (SV-1) 
Technical Architecture Profile (TV-1) 


Supporting Operational View Products 
Command Relationships Chart (OV-4) 
Activity Model (OV-5) 


Operational Activity Sequence and Timing Descrip- 


tions (OV-6a, 6b, and 6c) 
Logical Data Model (OV-7) 


Supporting Products Focused on System Evolution 
e« System Evolution Description (SV-8) 
.« System Technology Forecast (SV-9) 

. Standards Technology Forecast (TV-2) 


Table 4-1: Product Combinations 


CMU/SEI-99-TR-014 


Performance 
Dependability 


frequency of availability 
exchanges of assets 

timeliness 

throughput 

latency 


alternative 


§ 
throughput 
latency 
message size 


classification 
of messages 
encryptions 
exposure 
intersection 
spoofing 


Modifiability/ 
Interoperability 


standards 
protocols 
formats 
components 
contained 
functions 
performed 
formats 
LISI levels 
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Both use-case and change-case scenarios can be expressed at the level consistent with the 
C4ISR operational views or system views. Scenarios expressed at the level of the operational 
views can be further refined in terms of the more detailed system views, as described below: 


e Scenarios at the level of operational views will be expressed in terms of nodes, informa- 
tion flows, and activities. These can be analyzed against the operational views. 


¢ Scenarios at the level of the system views will be expressed in terms of systems, functions, 
and communication paths. These can be analyzed against the system views. 


For example, the following scenarios could be analyzed against operational views: 


¢ A use-case scenario may explore the impact of a node failure on a specific mission in a 
specific geographical region. 


¢ Achange-case scenario might explore the impact of adding a new communications satel- 
lite system to a mission. 


If the C4ISR operational view products are sufficiently detailed, the scenarios can be analyzed 
to determine their impact on the mission in terms of the nodes, command structure, activities, 
and communication paths. However, the scenarios’ impact on specific systems or functions 
cannot be determined at this level. In these cases, the scenarios must be refined in terms of sys- 
tems and functions and their connectivity, and then explored against the C4ISR system views. 
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Appendix A: C4ISR Products as Sources 
of Scenarios 


The C4ISR Framework distinguishes between essential and supporting products, and for each 
product it defines the information to be included in the product. These information tables are 
captured in the integrated dictionary, which is itself one of the products (AV-2). 


In this appendix, we identify items in the information tables that could be used to derive sce- 
narios. This is a literal scan of the product information tables, looking for implicit or explicit 


references to quality-attribute concerns. Further, we suggest how combining the views could 
provide coverage across multiple attributes. 


A.1 Essential Products 


A.1.1 Overview and Summary Information (AV-1) 


This view is both a planning guide and documentation upon completion of the development. 


It can be used to identify assumptions, constraints, attribute-specific requirements, and scenar- 
10S. 


A.1.2 Integrated Dictionary (AV-2) 
This is a central source for definitions and annotations to the graphical representations. 
It can be used to document 


¢  attribute-specific data (e.g., rates, probabilities, impact of changes) 


e risks identified during the architecture evaluation 
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A.1.3 High-Level Operational Concept Graphic (OV-1) 


A graphical representation of operations in terms of such things as missions, functions, organi- 
zations, and/or asset distribution. It describes weapon types, logical and physical connections, 
trajectories and targets, and criticality of mission. 


It can be used to assign weights to attribute-specific evaluations, sensitivities, and tradeoffs. 
Possible scenarios include 


¢ dependability (from points of failure and availability of assets) 
¢ performance (from throughput and latency of communications) 


* security (from exposure, interception, and spoofing) 


A.1.4 Operational Node Connectivity Description (OV-2) 


This describes the operational nodes, the need lines between them, and the characteristics of 
the information exchanged. Details appear in OV-3. 


A.1.5 Operational Information Exchange Matrix (OV-3) 


This matrix captures requirements for information exchanges between operational nodes. It 
adds details to OV-2. 


Possible scenarios include 
¢ interoperability (from protocols, formats and LISI levels) 


¢ security (from classification of messages) 
¢ dependability (from redundancy) 


¢ performance (from message size, media, frequency of exchange, timeliness, and through- 
put data) 


A.1.6 System Interface Description (SV-1) 


This description links together the operational and systems architecture views by depicting the 
assignments of specific systems and their interfaces to the nodes and need lines described in 
OV-2. See SV-2 for communication nodes. See SV-6 for system information elements 


Possible scenarios include 
¢ interoperability (from standards and protocols) 


* modifiability (from components contained and functions performed by each component) 
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e dependability (from redundancy of components/functions) 
¢ performance (from throughput, channel capacity, and latency data) 


¢ security (from encryption) 


A.1.7 Technical Architecture Profile (TV-1) 


This profile provides a time-phased enumeration of the relevant subset of technical standards 
that apply to the architecture and how they have been or are to be implemented. 


Possible scenarios include 
e interoperability (from protocols) 


¢ modifiability (from protocols and formats) 


A.2 Supporting Products 


A.2.1 Command Relationships Chart (OV-4) 


This chart illustrates the hierarchy of organizations or resources in an architecture and the rela- 
tionship among them (e.g., command, control, coordination). 


A possible scenario is dependability (from backups). 


A.2.2 Activity Model (OV-5) 


This model describes the applicable activities associated with the architecture, the data and/or 
information exchanged between activities, and the data and/or information exchanged with 
other activities outside the scope of the model (i.e., external interfaces). See OV-2 for opera- 
tional node and OV-3 for operational information element. 


A possible scenario is dependability (from flow of activities). 


A.2.3 Operational Activity Sequence and Timing Descriptions 
(OV-6a, 6b, and 6c) 

These descriptions consist of a set of three types of models needed to refine and extend the 

operational view, to adequately describe the dynamic behavior and performance characteris- 


tics of the business processes critical to an architecture. See OV-2 for operational nodes. These 
three models are described below: 
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¢ Operational rules model (OV-6a) — operational rules expressed in a formal language 


¢ Operational state transition description (OV-6b) — detailed timing sequence of activities 
or work flow in the business process 


¢ Operational event/trace description (OV-6c) — depiction of the dynamic behavior of mis- 
sion processes (used alone or in conjunction with OV-6b) 


Possible scenarios include 


¢ performance (from alternative paths, throughput, and latency) 
e dependability (from points of failure) 


A.2.4 Logical Data Model (OV-7) 


This model describes the data and information that are associated with the information 
exchanges of the architecture. 


Possible scenarios include 


¢ performance (from message size) 


¢ dependability (from value faults) 


A.2.5 Systems Communications Description (SV-2) 


This description represents the specific pathways or networks of the communications systems 
and the details of their configurations through which the physical node and systems interface. 
See SV-1 for systems nodes, systems, and links. See OV-2 for need lines and operational 
nodes. 


Possible scenarios include 


¢ performance (from WAN/LAN characteristics and steps in the path) 
¢ dependability (from fault rates and redundancy) 


¢ security (from classification, encryption and spoofing) 


A.2.6 Systems? Matrix (SV-3) 


This is a description of the system-to-system relationships identified in the various types (e.g., 
internodal and intranodal) in SV-1. See SV-6 for system information elements. 
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Possible scenarios include 


e interoperability (from existing/planned/potential/deactivated functionality) 


e security (from classification of data) 


A.2.7 Systems Functionality Description (SV-4) 


This describes the flow of data among system functions, and the relationship between systems 
or system functions and activities at nodes. See SV-1 for system function and system node. See 
SV-6 for system information element. 


This is not a likely source of scenarios. 


A.2.8 Operational Activity to System Function ἘΠΕ ΒΟΒΒΒΊΗΝ 
Matrix (SV-5) 


This matrix helps to link the operational and systems architecture views by depicting the 
“many-to-many” mappings of operational activities to systems functions. See SV-1 for system 
function. See OV-5 for operational activity. 


This is not a likely source of scenarios. 


A.2.9 System Information Exchange Matrix (SV-6) 


This matrix describes, in tabular form, the physical aspects of how the information exchanges 
called for in OV-2 actually are (or will be) implemented in terms of protocols, data formats, 
etc. 


This is particularly useful for understanding the potential for overhead and constraints intro- 
duced by these choices. See SV1 for system, system elements, system components, system 
functions. See SV-4 for the inputs to and outputs from the system elements. 


Possible scenarios include 


e interoperability (from media and format) 
e security (from classification) 


e performance (from frequency, timeliness, and throughput data) 
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A.2.10 System Performance Parameters Matrix (SV-7) 


This matrix builds on the “system element interface description” (SV-1) by portraying the cur- 
rent hardware and software performance characteristics of each system, and the or required 
performance characteristics at specified times in the future, geared towards TV-2. See SV-1 for 
system, system element, and system component. 


This product provides the following data that are needed for attribute-specific analyses: 


* mean time to failure (MTTF), maintainability, availability, system initialization time, data 
transfer rate, and program restart time for platforms 


¢ data throughput/capacity, response time, effectiveness, and mean time between software 
failures for application software 


Possible scenarios include 


¢ dependability (from MTTF for hardware and application software) 


¢ performance (from data transfer rate and throughput/capacity, system initialization, pro- 
gram restart, and response time) 


A.2.11 System Evolution Description (SV-8) 


This depicts how a suite of systems will be made more modern over time, including evolution 
and/or migration steps to accommodate the specific information requirements, performance 
parameters, and technology forecasts provided in other products. See SV-1 for system, system 
element, and system component. 


A possible scenario is modifiability (from upgraded functionality). 


A.2.12 System Technology Forecast (SV-9) 


This contains predictions about the availability of emerging technologies, specific hardware/ 
software products, and industry trends in short-, mid-, and long-term intervals, focused on 
technology areas that are relevant to the architecture’s purpose. 


A possible scenario is modifiability/interoperability (from upgraded capabilities). 


20 | CMU/SEI-99-TR-014 


A.2.13 System Activity Sequence and Timing Descriptions 
(SV-10a, 10b, and 10c) ) 
These descriptions consist of three types of models needed to refine and extend the systems 


view, to adequately describe the dynamic behavior and performance characteristics of the 
architecture. The three descriptions are described below: 


. Systems rules model (SV-10a) — constraints imposed on system functionality due to 
some aspect of system design or implementation, expressed in a formal language 


e Systems state transition description (SV-10b) — detailed sequencing of functions in a sys- 
tem | 


e Systems event/trace description (SV-10c) — description of dynamic behavior, tracing the 
actions in a scenario or critical sequence of events along a given time line (used alone or in 
conjunction with SV-10b) 


This product may reflect system-specific aspects or refinements of critical sequences of events 
described in the operational view (e.g., performance-critical scenarios). See OV-6a, 6b, and 6c 
for the sequences of events. 


Possible scenarios include 


¢ modifiability (from removing constraints on functionality) 


¢ performance (from execution paths and event times) 


dependability (from points of failure) 
¢ security (from points of attack) 


A.2.14 Physical Data Model (SV-11) 


᾿ς Describes how the information represented in OV-7 is actually implemented (that is, how the 
information exchange requirements actually are implemented and how both data entities and 
their relationships are maintained in the systems architecture). 


Possible scenarios include 


¢ performance (from message size) 


¢ dependability (from file system properties) 


A.2.15 Standards Technology Forecast (TV-2) 


This consists of detailed descriptions of emerging technology standards and implementing 
products that are relevant to the systems and business processes covered by the architecture in 
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short-, medium-, and long-term intervals, with confidence factors for the prediction, along 
with issues that may affect the architecture. See TV-1 and SV-9 for the timeline of the stan- 
dards and the prediction of availability of the emerging technologies. 


A possible scenario is modifiability/interoperability (from new standards). 


A.3 Relationships Between C4ISR Products 
[Source: product attribute tables in Appendix A of the C4ISR Framework.] 


implied entities, attributes & 
Gets details from relationships 
Overview and Summary OV-1 
Information (AV-1) . Organization 
2) 


High-Level Operational OV-3 
wane rap me (OVI): +e Operational Information 
| Element 


Operational Node Con- 
nectivity Description (OV- 
2) 


¢ Operational Information Element 
OV-5 


System Interface Descrip- | SV-2 OV-2 

tion (SV-1) ¢ Communication Node ° Need line 
Technical Architecture 

Profile (TV-1) 


Table A-1: Detailed Observations About Essential Products 


OV-2 
e Need Itne 

¢ Operational Node 
OV-5 | 
° Activity 


Operational Information 
Exchange Matrix (OV-3) 


"tn 
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Implied entities, attributes & 


Gets details from relationships 


Command Relationships OV-1 
Chart (OV-4) | ¢ Organization 
Activity Model (OV-5) OV-2 

e Operational Node 


OV-2 
e Operational Node 


Operational Activity 
Sequence and Timing 


Descriptions (OV-6a, 6b, 
and 6c) 


OV-2 
¢ Need line 

e Operational Node 

SV-1 

¢ Systems node contains system 


SV-1 
¢ — Systems Node 
« System 

e Link 


Systems Communications 
Description (SV-2) 


e Operational node maps to systems node 
e Link implements need line 


SV-1 
e System 

SV-6 

e System Information Element 


Systems Matrix (SV-3) 


SV-6 
e — System Information Element 


SV-1 
e Systems Function 
e Systems Node 


Systems Functionality 
Description (SV-4) 


SV-1 
e System Function 
OV-5 

¢ Operational Activity 
e System Function 


Operational Activity to 
System Function Trace- 
ability Matrix (SV-5) 


Table A-2: Detailed Observations About Supporting Products 
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implied entities, attributes & 
relationships 


Gets details from 


System Information 
Exchange Matrix (SV-6) 


e System, System Element 


¢ Application Software (System Compo- 
nent) 


e¢ System Function 


¢ System performs system function 


¢ System element performs system func- 
tion 


System information element is input to 
system function 


¢ System is source of system information 
element 


System Performance 
Parameters Matrix (SV-7) 


¢ System, System Element 


¢ Application Software (System Compo- 
nent) 


« System contains system element 


System element contains system compo- 


System Evolution 
Description (SV-8) 


System Technology Fore- 
cast (SV-9) 


System Activity Sequence 
and Timing Descriptions 
(SV-10a, 10b, and 10c) 


System Element 


System Component 


Physical Data Model (SV- 
11) 


Standards Technology 
Forecast (TV-2) 


Reference Model 


Service Area, Service 


Reference model includes service area 


Service area includes service 


Table A-2: Detailed Observations About Supporting Products (Continued) 
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Appendix B: Screening Questions for the 
C4ISR Operational View 


B.1 Operational Environment 


B.1.1 Mission and Geography 


¢ Which missions in which geographical regions would you expect to place stress on the 
system resources? 


¢ How would you define the workload of each such stressful situation? 
¢ What assets would you expect to be deployed? | 


When these have been answered and follow-up questioning and explanations have been com- 
pleted, we should have an operational picture of the expected stressful situation. Presumably 
the architect would use the operational views to represent this picture. This will help us to 
understand what is going on. 


B.1.2 Organization and Command Structure 
.« What is the command structure for the mission/geographical region? 


¢ How does this command structure interweave with the U.S. Coast Guard (USCG) opera- 
tional hierarchy? 


e Can this structure change with asset or communications failure? 


B.1.3 Quality Attributes 


B.1.3.1 Dependability 


¢ When the assets are deployed on a mission, are there different ways in which the commu- 
nications system can fail (for example, going out of line of site, equipment failure)? 


e Are their redundant communication paths, and how is reconfiguration achieved on failure? 


¢ Are their particular assets whose failure would cause the mission to be degraded or aban- 
doned? 
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¢ What types of fixes to failed equipment are done during the mission? Are there trained 
personnel and spare parts on board? 


e Are there cases where faults can propagate causing widespread failures? 


B.1.3.2 Performance 
Bandwidth 


¢ How much information will be transmitted over the communications paths and at what 
intervals? 


¢ Are there bursts of information that could saturate the communications paths and cause 
delays to critical information? 


¢ Are there large messages being transmitted, which could interfere with other traffic (e.g., 
initialization, charts)? 


Response Times 


¢ Do operational personnel require some critical responses, and what type of delay is 
acceptable? Have you performed an analysis to ensure that this can be met under high- 
stress burst conditions within reason? 


e Are there some critical response times to detected trigger events? 


B.1.3.3Security 


e Are there missions where the enemy will be attempting to create false alarms to confuse 
the mission? How can you prevent these false alarms from affecting the mission? 


e [5 there critical information that must be consistent and have protected access against 
unauthorized disclosure? 


¢ Is multi-level access necessary? 


e Js denial of service an issue? 


B.2 Logistics Environment 


B.2.1 Platform and Command Center Upgrades 
¢ Which anticipated platform upgrades will be most challenging for team formation and 
mission operation? 


¢ Which asset upgrades will happen together (all operational assets of Type A will be 
removed from service, upgraded, and restored to service together), and which will be done 
on a one-at-a-time basis? 
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¢ Will different types of upgrades have different strategies (e.g., new functionality, technol- 
ogy refreshment, problem fixes)? 


B.2.2 Evolution Strategy 

e [5 there a policy about keeping technology relatively fresh? 
¢ Will components be sustained beyond their end of life? 

e Is there a policy about installing obsolete technology? 


B.2.3 Legacy Systems 
e Are legacy systems to be upgraded as part of the plan? 
e How will the legacy system upgrades be accomplished? 


B.2.4 Quality Attributes 


B.2.4.1 Modifiability | 
e Is there a list of expected changes that will occur to the system during its life expectancy 
(e.g., threats, technology, operational, budget, schedule)? 


e How have these predicted changes affected the system architecture? 
e Have contingency plans been developed based on these predicted changes? 


¢ Are the requirements likely to remain stable or to change intrusively? 


B.2.4.2Inter-operability 


e Are there well-defined procedures for developing interfaces to external agencies? 
e Are all such interfaces defined by some standard? 

e Is there a plan to configuration manage the interfaces? 

e Is there acommon data model for interfaces shared by all participating agencies? 


The above questions could also be important for different integrated deep-water system (IDS) 
assets. 
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Appendix C: Example ATAM Scenarios 


The following example scenarios were generated during actual ATAM evaluations. 


Ο.1 Scenarios About the Development Project 
Scenario [Stakeholder(s); Quality/Qualities] 


Decrease cost of development by 20%. [Customer, project manager; cost, ability to subset] 


Project manager adds two new developers in a way to make them productive quickly. [Devel- 
oper, project manager; buildability, conceptual integrity] . 


Implement a skeletal version of the system first. Add functionality incrementally. [Project 
manager, developers, integrators, testers; ability to subset] 


An error is discovered during testing. The development organization identifies the source of 
the error and corrects it. [Developer, maintainer; maintainability, testability] 


Schedule the threads and processes; verify that the set is schedulable. [Developer, performance 
engineer; performance, schedulability] 


The network is overloaded (for network-based systems). Re-partition tasks to reduce traffic. 
(Developer, performance engineer; performance, schedulability] 


Predict priority, execution time, memory space usage, deadline satisfaction, and resource con- 
tention when adding new software. [Developer, performance engineer; schedulability, concep- 
tual integrity] 


Produce a version of the system that runs on a desktop machine (for systems that run on spe- 
cialized platforms or are embedded). [Developers; portability] 
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C.2 Scenarios About the System During Operation 
Scenario [Stakeholder(s); Quality/Qualities] 


One of the outputs is erratic. Track down and fix the bug. [Tester, maintainer; testability, mod- 
ifiability] 


An error (€.g., divide by zero) occurs during processing. How does the system detect it, report 
it, handle it, and/or correct it? [User, tester, developer, maintainer, field support; conceptual 
integrity, testability] 


System initialization and start-up occurs. How does the architecture handle it? Perhaps appli- 
cation code must run before system start-up is complete. [Developer, user, maintainer; concep- 
tual integrity, modifiability] 


Allow a user to access the system remotely (e.g., via the Web, via radio signals, via dial-up). 
[User; modifiability, security] 


Verify the system’s computations, and/or the timing of those computations, at runtime. [Tester, 
maintainer, performance engineer; modifiability, fault tolerance, reliability] 


One processor or server goes down during operation. Half of the processors or servers go 
down during operation. The network fails during operation. One or more input devices fail 
during operation. [User; fault tolerance, availability] | 


Transmit data (to other systems, or across the network in this system) in encrypted form. 
[User; security] 


If using a distributed system, change the processor on which a specific piece of software runs. 
[User, maintainer, performance engineer; fault tolerance, scalability, portability] 


If using a distributed system, use a fault tolerance scheme that accounts for software migration 
off of a failed or damaged processor. [Maintainer; modifiability, fault tolerance, availability] 


Degraded operation mode: A user wants to continue to operate the system (perform useful 
work) in the presence of one of the following kinds of failures: database server, application 
server, or partial network failure. [User, maintainer; fault tolerance, availability, modifiability] 


Reduce system start-up (or re-start) time by a given factor. [User, performance engineer; mod- 
ifiability, performance] 
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An algorithm fails; system must revert back to using a previous (and more reliable) version. 
[User; fault tolerance] 


Only certain users can access certain data in the database. [User; modifiability, security] 


Have the system operate for 24 hours a day, 7 days a week. [User; reliability] 


C.3 Scenarios About Maintaining or Upgrading the 
System 


Scenario [Stakeholder(s); Quality/Qualities] 
Add a new data server. [Maintainer; modifiability, performance] 


Replace one of the input devices with one of similar or slightly enhanced capability. [Main- 
tainer; modifiability] 


A new input device replaces five current ones. [Maintainer; modifiability] 


Performance requirements are doubled. Data throughput is doubled. [Maintainer, performance 
engineer; performance, modifiability] 


The number of users and/or the number of inputs doubles. [Maintainer, perience engineer; 
performance, modifiability, scalability] 


The requirement to use a particular commercial off-the-shelf (COTS) component is levied. 
[Maintainer; openness, modifiability] 


The computer is replaced. If there is more than one computer, heterogeneous computers are 
introduced. The compiler is replaced. The operating system is replaced. The network or bus is 
replaced. [Maintainer; modifiability, portability] | 


Add a completely new function; compute a new result. [Customer, user; modifiability, concep- 
tual integrity] 


Make the system interact with some other (specified) system in some (specified) way. (For 
example, have the system participate in a cross-system simulation exercise.) [User, users of 


other system; modifiability] 


Introduce a new data format or message format. [User, users of other system; modifiability] 
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Introduce a recording/playback capability. [User; modifiability] 


Replace the middleware or communications infrastructure with one supplied by a different 
vendor. [Customer, performance engineer; modifiability] 


Add automated decision aids. [User; modifiability] ὁ 
Replace the database. Add an object-oriented database. [Developer, customer; modifiability] 
Replace the user interface. [User, developer; modifiability] 


Introduce data from a completely new source; introduce real-time data. [User, customer; mod- 
ifiability, performance] 


C.4 Scenarios Specifically About a Product Line 
Architecture 


Scenario [Stakeholder(s); Quality/Qualities] 


A product developer wants to build the smallest possible running system (using only the archi- 
tecture and some subset of the reusable assets of the product line) that computes some useful 
result. [Product line application developer; applicability of architecture to product line, con- 
ceptual integrity, ability to subset] 


A new reuser (subscriber) wishes to join the product line, but needs to know if his performance 
and accuracy requirements can be met using the core assets. [Reuser, (new) product line appli- 
cation developer; applicability of architecture to product line, conceptual integrity, ability to 
subset] 


Issue a new release of the core asset library. [Product line application developers; applicability 
of architecture to product line] 


An error is found in a member of the product line, in the field. How is the error corrected, and 
how does it affect the core assets and other members of the product line? [Testers, service rep- 
resentatives; ability of architecture to isolate errors] 


The domain (scope) of the product line is expanded to include. . . . [Product line manager; 
marketers; product line application developers; flexibility, applicability of architecture to 
scope, modifiability] 
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